Software systems such as Enterprise Resource Planning (ERP) systems are the computational backbone of many business organizations. Such software systems often include a large number of software modules that need to be frequently validated to ensure that the software systems are operating as expected. This validation typically involves extensive testing, which can be an expensive and labor-intensive process. In particular, configuration changes to a software system may be risky and require regression testing to validate affected configuration elements (e.g., business processes impacted by the changes). The cost and time of a full regression test may be too high.
Determining what tests to run in order to validate configuration elements is not a trivial task. Often, configuration elements may have unintended and/or unexpected dependencies and/or side effects on a software system that a tester may not be aware of. Since tests are usually designed and run by each organization separately on the organization's systems, this means that each organization must build up the necessary knowledge and independently discover important aspects that should be tested when validating different configuration elements. Gaining this knowledge may require much effort and experience; in the meantime, testing the software systems may be a less effective and prolonged process. However, organizations that are able to utilize each other's testing-related knowledge, which is in a sense a wisdom of the crowd (of testers), may be able to come up with a more effective and efficient testing plan.
Collecting testing data from multiple organizations may not be that helpful if there is no way to sift through the massive amounts of data that can be collected and select appropriate tests for the task at hand. Additionally, not all tests are equal. While some organizations may have knowledge of how to test certain aspects of a software module (e.g., effective tests for testing a certain configuration element), others may have devised erroneous, ineffective, and/or irrelevant tests.